fix(session): retry HTTP 408 request timeouts - #39413
Conversation
retryable() only bypasses the provider SDK's isRetryable flag for status >= 500, so a 408 the SDK did not mark retryable ended the turn and the user had to resend the prompt. OpenAI-compatible proxies that normalize an aborted upstream stream into 408 request_timeout hit this. 408 is the one 4xx that is transient in the same way a 5xx is: the request timed out before completing, so it is safe to send again. Treat it exactly like the existing 5xx case, same backoff and retry-after handling. Other 4xx statuses are unchanged. Closes anomalyco#39221
|
The following comment was made by an LLM, it may be inaccurate: Potential Duplicate FoundPR #38683: fix(session): retry transient transport failures This PR appears related because it also addresses retrying transient failures in the session layer. Since the current PR (39413) specifically handles HTTP 408 request timeouts as a transient failure, it's worth verifying that #38683 doesn't already cover this case or if there's overlap in approach. |
|
Different layer, and #39221 calls this out explicitly: #38683 (closes #30611) classifies raw/wrapped transport failures by walking the cause chain in They do land near each other in |
|
Please merge as soon as possible. |
|
#39221 was closed as addressed by #39391, which I'd missed — that one adds 408/409 to This PR touches a different spot: Happy to close this if the v1 session retry path is on its way out and only the |
|
I opened follow-up PR #40268 for a separate failure shape observed with CLIProxyAPI: the HTTP response is 200, but the Responses SSE stream contains a top-level In local logs, three such failures were each followed by a new stream attempt, and the session completed without a user-visible error after the retry path was fixed. #40268 references #39221 as a follow-up and intentionally does not close it. |
|
Thanks @fashen97 — #40268 looks complementary rather than overlapping, and the two cover different classification points:
Your version sets On merge order: we both touch Worth noting for whoever picks these up: #39221 was closed as addressed by #39391, which fixed 408/409 in |
Issue for this PR
Closes #39221
Type of change
What does this PR do?
retryable()only bypasses the provider SDK'sisRetryableflag forstatus >= 500. An HTTP 408 that the SDK didn't mark retryable therefore falls through and ends the turn, so the user has to resend the prompt by hand. This shows up with OpenAI-compatible proxies that normalize an aborted upstream stream into408 request_timeout.408 Request Timeout is the one 4xx that is transient in the same way a 5xx is — the request never completed, so sending it again is the defined behavior for that status. This adds 408 to the existing condition, so it takes exactly the same path as 5xx: same classification, same backoff, same
retry-afterhandling. No new retry mechanism, and other 4xx statuses are unchanged.How did you verify your code works?
Added
retries 408 request timeout errorstotest/session/retry.test.ts, built from the payload in the issue (status 408,isRetryable: false,code: request_timeout), asserting it is now classified retryable. Ran the wholeretry.test.tssuite so the neighbouring cases still hold — in particulardoes not retry 4xx errors when isRetryable is false(status 400) still passes, so this doesn't loosen 4xx generally.bun typecheckpasses.Checklist